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A  United  States 

I  -w'/AI  I  General  Accounting  Office 
Washington,  D.C.  20548 

Resources,  Community,  and 
Economic  Development  Division 

B-245928 

October  30,  1991 

The  Honorable  John  Glenn 
Chairman,  Committee  on  Governmental 
Affairs 

United  States  Senate 
Dear  Mr.  Chairman: 

In  response  to  your  January  24,  1991,  request,  this  report  discusses  the  Department  of 
Energy’s  (doe)  appraisal  process  for  management  and  operating  contractors.  The  issues 
discussed  include  (1)  the  effectiveness  of  doe’s  use  of  performance  objectives  to  set 
expectations  and  evaluate  contractor  performance,  (2)  the  effectiveness  of  doe’s  use  of  data 
from  on-site  reviews  to  evaluate  contractor  performance  for  award  fee  purposes,  and  (3)  the 
effect  of  doe’s  new  award  fee  regulations  on  the  performance  evaluation  and  award 
determination  process. 

As  arranged  with  your  office,  unless  you  publicly  announce  its  contents  earlier,  we  plan  no 
further  distribution  of  this  report  until  30  days  from  the  date  of  this  letter.  At  that  time,  we 
will  send  copies  to  the  Secretary  of  Energy;  the  Director,  Office  of  Management  and  Budget; 
and  other  interested  parties. 

This  work  was  performed  under  the  direction  of  Victor  S.  Rezendes,  Director,  Energy  Issues, 
who  may  be  reached  at  (202)  275-1441.  Other  major  contributors  are  listed  in  appendix  I. 


Sincerely  yours, 

J.  Dexter  Peach 

Assistant  Comptroller  General 


Elxecutive  Summary 


Purpose 


GAO  has  recently  implemented  a  special  audit  effort  to  help  ensure  that 
areas  vulnerable  to  fraud,  waste,  abuse,  and  mismanagement  are  identi¬ 
fied  and  that  appropriate  corrective  actions  are  taken.  This  effort 
focuses  on  16  areas,  one  of  which  is  the  Department  of  Energy’s  (doe) 
contracting  practices.  As  part  of  this  special  audit  effort,  the  Chairman, 
Senate  Committee  on  Governmental  Affairs,  requested  that  gao  examine 
doe’s  process  for  formally  reviewing  and  assessing  the  performance  of 
management  and  operating  (m&o)  contractors.  Specifically,  gao 
examined  (1)  the  effectiveness  of  doe’s  use  of  performance  objectives  to 
set  expectations  and  evaluate  contractor  performance,  (2)  the  effective¬ 
ness  of  doe’s  use  of  data  from  on-site  reviews  to  evaluate  contractor  per¬ 
formance  for  award  fee  purposes,  and  (3)  the  effect  of  doe’s  new  award 
fee  regulations  on  the  performance  evaluation  and  award  determination 
process. 

As  agreed  with  the  requester,  gao  focused  specifically  on  those  contrac¬ 
tors  operating  under  cost-plus-award-fee  contracts  with  doe’s  Albu¬ 
querque  Field  Office.  This  office  was  selected  because  it  administers 
contracts  for  four  large  defense  materials  production  plants — Kansas 
City,  Mound,  Pantex,  and  Pinellas — as  well  as  the  contract  for  the  Waste 
Isolation  Pilot  Plant  (wipp),  in  which  doe  plans  to  store  radioactive 
waste. 


Background 


doe’s  m&o  contracts  with  profit-making  companies  are  generally  cost- 
plus-award-fee  contracts.  Twenty-nine  of  doe’s  52  M&o  contracts  are 
cost-plus-award-fee  contracts.  The  fee  consists  of  a  fixed  amount  (called 
the  “base  fee”)  and  an  award  amount  (called  the  “award  fee  pool”)  that 
is  available  to  the  contractor  for  performance  above  minimally  accept¬ 
able  levels,  doe  evaluates  the  contractor’s  performance  every  6  months 
to  determine  the  amount  of  the  award,  if  any.  In  fiscal  year  1990,  the 
potential  award  amounts  ranged  from  $4.6  million  for  the  wipp  con¬ 
tractor  to  $12.1  million  for  the  Kansas  City  Plant  contractor. 


Results  in  Brief 


The  Albuquerque  Field  Office’s  (Albuquerque)  use  of  performance 
objectives  does  not  result  in  effective  evaluations  of  contractor  perform¬ 
ance  or  effectively  communicate  doe’s  expectations  to  the  contractors. 
Albuquerque’s  contractor  performance  objectives  are  often  very  broad 
and  provide  no  criteria  or  standards  against  which  to  evaluate  con¬ 
tractor  performance.  As  a  result  of  this  lack  of  specificity,  judgments 
about  performance  are  subjective.  A  February  1991  doe  report  on  a  task 
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force  study  of  the  award  fee  process  recommended  building  more  objec¬ 
tivity  into  the  performance  evaluation  plans  and  including  performance 
indicators  that  are  as  specific  as  possible.  To  successfully  implement 
such  changes,  however,  doe  must  change  its  traditional  philosophy  of 
establishing  broad  objectives  and  relying  on  its  contractors  to  plan  and 
carry  out  the  programs. 

Albuquerque  does  not  have  procedures  and  controls  in  place  to  ensure 
that  all  contractor  performance  data  are  appropriately  evaluated  as 
part  of  the  award  process.  Although  various  on-site  reviews  of  specific 
contractor  functions,  such  as  procurement  or  financial  management,  are 
periodically  performed,  the  results  of  the  reviews  have  sometimes  been 
discounted  in  performance  evaluations  and  award  determinations.  Fur¬ 
thermore,  the  evaluations  and  award  determinations  do  not  consistently 
reflect  the  contractors’  lack  of  responsiveness  to  review  findings,  even 
those  findings  that  are  long-standing.  Finally,  even  though  doe  requires 
contractors  to  prepare  self-assessments  that  include  the  status  of  previ¬ 
ously  identified  deficiencies,  Albuquerque  has  not  enforced  contractor 
compliance  with  this  requirement. 

doe’s  new  award  fee  policy  is  designed  to  both  penalize  contractors  for 
avoidable  costs  such  as  the  loss  or  destruction  of  property  and  provide 
substantially  higher  compensation  for  good  management  practices.  Suc¬ 
cessful  implementation  of  the  policy  on  use  of  penalties  will  be  difficult 
until  DOE  develops  new  management  procedures  and  provides  technical 
training  for  its  staff  so  that  it  can  identify  and  hold  contractors  account¬ 
able  for  these  costs.  As  a  consequence,  contractors  may  realize  substan¬ 
tial  increases  in  compensation  before  doe  is  able  to  enforce  the  rule’s 
provisions. 


Principal  Findings 


Broad  Objectives  Hamper 
the  Effectiveness  of 
Contractor  Evaluations 


Specific  and  measurable  performance  objectives  are  important  to 
decrease  the  subjectivity  of  the  evaluation  process.  Many  objectives  that 
Albuquerque  has  established  for  its  m&o  contractors,  however,  are  very 
broadly  stated  and  contain  no  criteria,  standards,  or  milestones  against 
which  to  measure  contractor  performance.  For  example,  one  perform¬ 
ance  objective  for  the  Mound  Plant  was  to  “enhance  the  Industrial 
Hygiene  Program.”  With  such  broad  objectives,  evaluations  of  con¬ 
tractor  performance  are  highly  subjective,  causing  disagreements  and 
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confusion  on  the  part  of  doe  units  as  well  as  the  contractors  about  how 
to  assess  performance. 

Although  DOE  is  moving  toward  requiring  more  specific  objectives, 
developing  and  enforcing  these  objectives  constitutes  a  major  change  in 
the  way  doe  has  done  business  with  its  contractors.  The  February  1991 
DOE  task  force  report  to  the  Assistant  Secretary  for  Defense  Programs, 
which  recommended  more  specificity  in  performance  evaluation  plans, 
stated  that  some  doe  officials  continue  to  believe  that  performance 
objectives  need  to  be  general  to  allow  contractors  the  flexibility  to  apply 
their  management  expertise  and  respond  to  changing  events.  This  belief 
reflects  the  traditional  way  that  doe  has  managed  its  contractors — that 
is,  by  establishing  broad  objectives  and  relying  on  contractors  to  plan 
and  carry  out  the  programs. 


In  preparing  contractor  evaluations,  doe  officials  are  to  consider  all  per¬ 
tinent  information,  such  as  findings  from  previous  evaluations  and  on¬ 
site  reviews.  However,  gao  found  that  significant  findings  that  Albu¬ 
querque  noted  in  some  on-site  reviews  were  discounted  in  the  contrac¬ 
tors’  performance  evaluation  reports.  For  example,  a  fiscal  year  1989 
Contractor  Purchasing  System  Review  at  wipe  revealed  significant 
weaknesses,  but  these  were  not  cited  as  deficiencies  in  the  contractor’s 
performance  evaluation  report. 

In  addition,  Albuquerque  does  not  ensure  that  contractors  take  prompt 
action  to  correct  identified  deficiencies.  Three  of  the  10  Albuquerque 
divisions  whose  review  procedures  we  examined  have  not  established 
procedures  specifying  when  and  how  contractors  are  to  address  review 
recommendations.  Six  of  the  divisions  have  not  established  central  sys¬ 
tems  to  track  contractors’  corrective  actions.  Accordingly,  a  contractor’s 
failure  to  implement  recommendations  not  only  may  go  unnoticed  by 
Albuquerque  officials  but  also  may  not  affect  the  contractor’s  perform¬ 
ance  evaluation.  For  example,  although  Albuquerque  had  not  ensured 
that  corrective  action  had  been  taken  on  14  of  16  recommendations 
made  6  years  earlier  in  a  personal  property  review  of  the  Pinellas  con¬ 
tractor,  this  fact  was  not  considered  in  the  contractor’s  performance 
evaluation  for  the  first  half  of  fiscal  year  1990.  In  fact,  Albuquerque 
considered  the  contractor’s  performance  in  the  personal  property  area 
to  be  a  “significant  achievement.” 

doe  established  a  new  process  in  fiscal  year  1990  to  help  monitor  con¬ 
tractors’  corrective  actions.  Contractors  are  required  to  submit  self- 


Albuquerque  Lacks 
Procedures  to  Ensure 
That  Deficiencies  Are 
Considered  and  Promptly 
Corrected 
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assessments  that  include  the  status  of  all  prior  deficiencies,  doe  can 
reduce  the  awards  if  contractors  do  not  submit  adequate  self-assess¬ 
ments.  Although  the  Albuquerque  contractors’  self-assessments  have 
generally  not  reported  the  status  of  previous  deficiencies,  Albuquerque 
officials  have  not  yet  penalized  any  contractors. 

DOE  Faces  Major 
Challenges  in 
Implementing  Its  New 
Accountability  Rule 

doe’s  new  rule  to  increase  contractor  accountability  is  designed  to  make 
the  contractors,  rather  than  the  government,  accountable  for  costs  that 
could  have  been  avoided  by  prudent  contractor  action.  However,  doe 
faces  formidable  challenges  in  implementing  this  requirement.  For 
example,  to  identify  contractor  accountability  for  loss  or  destruction  of 
government  property,  doe  will  need  to  develop  procedures  for  such 
things  as  verifying  the  accuracy  of  the  physical  inventories  conducted 
by  the  contractors. 

Furthermore,  to  compensate  the  contractors  for  their  increased  risk,  the 
rule  also  provides  for  substantially  higher  compensation.  Because  of  the 
magnitude  of  the  changes  needed,  contractors  may  be  eligible  for  the 
increased  compensation  long  before  doe  is  able  to  fully  implement  the 
accountability  requirements. 

Recommendations 

GAO,  among  other  things,  recommends  that  the  Secretary  of  Energy  take 
actions  to  ensure  the  development  of  (1)  specific,  measurable  perform¬ 
ance  objectives;  (2)  procedures  that  appropriately  reflect  the  results  of 
on-site  reviews  in  performance  evaluations  and  that  track  contractors’ 
responses  to  previously  identified  deficiencies  and  consider  these 
responses  in  the  evaluations;  and  (3)  procedures  and  staff  training 
needed  to  implement  the  new  accountability  rule  as  a  prerequisite  for 
implementing  the  planned  increase  in  contractor  compensation. 

Agency  Comments 

As  requested,  GAO  did  not  obtain  written  agency  comments  on  this 
report.  However,  the  information  in  this  report  was  discussed  with 
agency  officials,  who  agreed  with  the  factual  information.  Their  views 
were  included  where  appropriate. 
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The  Department  of  Energy  (doe)  uses  management  and  operating  (m&o) 
contracts  for  the  operation,  maintenance,  or  support  of  DOE-owned 
research,  development,  production,  and  testing  facilities.  All  of  doe’s 
M&o  contracts  are  fully  reimbursable,  cost-type  contracts.  A  few  of  these 
contracts  provide  for  a  fixed  fee,  while  the  contracts  with  nonprofit 
organizations  are  generally  no-fee  contracts.  However,  the  majority  of 
doe’s  m&o  contracts — 29  out  of  52  M&o  contracts — are  cost-plus-award- 
fee  contracts. 

A  cost-plus-award-fee  contract  is  a  cost-reimbursement  contract  with 
special  fee  provisions  consisting  of  a  fixed  amount  (called  the  “base 
fee”)  and  an  award  amount  (called  the  “award  fee  pool”).  The  base  fee 
does  not  vary  with  performance;  rather,  it  is  meant  to  compensate  the 
contractor  for  factors  such  as  risk,  investment,  and  the  nature  of  the 
work  performed.'  The  award  fee  pool  is  the  amount  of  money  poten¬ 
tially  available  to  the  contractor  for  performance  above  minimally 
acceptable  levels.  The  amount  of  the  award,  if  any,  is  determined  by 
doe’s  evaluation  of  the  contractor’s  performance,  doe  uses  award  fee 
contracts,  among  other  things,  to  encourage  effective  work  by  contrac¬ 
tors  and  improve  the  quality  of  their  performance. 

doe’s  Albuquerque  Field  Office  (Albuquerque),  where  we  concentrated 
our  review,  is  central  to  the  management  of  the  nuclear  weapons  com¬ 
plex.  Through  cost-plus-award-fee  contracts,  Albuquerque  manages  and 
operates  four  of  the  nation’s  seven  nuclear  weapons  production  plants: 
the  Kansas  City  Plant;  Mound  Plant  (in  Miamisburg,  Ohio);  Pantex  Plant 
(near  Amarillo,  Texas);  and  Pinellas  Plant  (near  St.  Petersburg,  Florida). 
Albuquerque  also  administers  the  cost-plus-award-fee  contract  for  the 
management  and  operation  of  the  underground  Waste  Isolation  Pilot 
Plant  (wipp)  (near  Carlsbad,  New  Mexico),  in  which  doe  plans  to  store 
radioactive  waste. 


'As  discussed  in  ch.  4,  under  DOE’s  revised  regulations,  half  of  the  base  fee  is  at  risk  if  contractor 
performance  is  not  satisfactory. 
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GAO  and  DOE  Have 
Recognized  the  Need 
to  Improve  DOE 
Contractor 
Management 


doe’s  oversight  of  its  contractors  is  1  of  16  areas  we  have  identified  as 
being  highly  vulnerable  to  mismanagement,  fraud,  and  abuse.  We 
included  doe’s  contract  oversight  as  a  high-risk  area  in  part  because  doe 
had  provided  awards  to  contractors  in  questionable  situations.  For 
example,  our  October  1989  report  on  awards  paid  to  the  Rocky  Flats, 
Colorado,  contractor  disclosed  that  the  contractor  received  millions  of 
dollars  in  awards  despite  significant  problems  in  its  environment,  safety, 
and  health  (es&h)  programs.2 

The  Secretary  of  Energy  has  also  recognized  the  need  to  improve  doe’s 
contractor  oversight  process.  In  a  September  1989  memorandum  to  all 
doe  personnel,  the  Secretary  stated  that  contract  management  within 
the  Department  needed  special  corrective  action.  Specifically,  he  noted 
that  DOE  needed  to  modify  and  significantly  strengthen  existing  con¬ 
tracting  strategies,  particularly  in  the  area  of  compensation  manage¬ 
ment.  Among  the  contract  management  improvements  that  the 
Secretary  said  were  needed  were  (1)  expanded  incentives  for  contrac¬ 
tors  to  achieve  excellence  and  cost  effectiveness  in  their  performance, 
(2)  an  enhanced  understanding  of  performance  expectations  and  per¬ 
formance  criteria  by  both  federal  and  contractor  employees,  and  (3) 
tighter  controls  to  ensure  that  doe  line  managers  take  corrective  action 
when  contractors  do  not  perform  to  standards. 

In  addition,  in  doe’s  fiscal  year  1989  and  1990  reports,  required  by  the 
1982  Federal  Managers’  Financial  Integrity  Act,  the  Secretary  disclosed 
contract  management  as  a  material  weakness.  Both  reports  stated  that 
performance  expectations  and  performance  criteria  needed  to  be  better 
defined  and  that  oversight  of  both  prime  contracts  and  subcontracts 
needed  to  be  improved. 


DOE  Has  Taken 
Actions  to  Improve 
the  Award  Process 


DOE  has  recently  taken  several  actions  to  improve  the  award  process. 

DOE  (1)  increased  its  emphasis  on  es&h  problems,  (2)  formalized  a  policy 
to  withhold  a  contractor’s  entire  award  if  performance  in  any  one  major 
functional  area  is  unsatisfactory,  and  (3)  issued  new  regulations  holding 
contractors  accountable  for  certain  costs  that  have  historically  been 
borne  by  the  government. 

Since  June  1989,  at  the  direction  of  the  Secretary  of  Energy,  doe  has 
based  at  least  51  percent  of  the  contractors’  award  fees  on  compliance 


^Nuclear  Health  and  Safety:  DOE’s  Award  Fees  at  Rocky  Flats  Do  Not  Adequately  Reflect  ES&H 
Problems  (0AO/RCED-9U-47,  Oct.  23,  1989). 
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with  ES&H  regulations.  This  new  emphasis  on  es&h  has  already  resulted 
in  reduced  award  fees  for  Albuquerque’s  five  award  fee  contractors. 
Awards  decreased  for  all  five  contractors;  for  some  the  decrease  was 
substantial.  Table  1.1  compares  the  percent  of  available  award  fee  pools 
paid  to  the  Albuquerque  contractors  in  fiscal  years  1989  and  1990. 


Table  1.1:  Percent  of  Available  Award 
Fee  Pools  Paid  to  Albuquerque 
Contractors,  Fiscal  Years  1989  and  1990 


Facility 

Fiscal  year  1989 

Fiscal  year  1990 

1st  half 

2nd  half 

1st  half 

2nd  half 

Kansas  City  Plant 

81.3 

68.5 

61.6 

78.4 

Pantex  Plant 

79.2 

59.5 

53.6 

56.0 

Mound  Plant 

76.4 

67.4 

56.8 

49.5 

Pinellas  Plant 

89.7 

85.5 

57.2 

64.4 

WIPP 

87.7 

85.3 

54.3 

57.6 

DOE  has  also  formalized  its  policy  that  allows  the  fee  determining  official 
to  withhold  a  contractor’s  entire  award  for  the  evaluation  period  when 
the  contractor’s  performance  is  unacceptable  in  any  one  of  the  major 
performance  areas.  Since  the  policy  was  formalized  in  November  1989, 
no  Albuquerque  contractor  has  been  rated  unacceptable  in  any  one  of 
the  major  performance  areas,  and  Albuquerque  has  not  imposed  this 
penalty. 

In  February  1991  doe  published  interim  regulations  to  hold  the  contrac¬ 
tors,  rather  than  the  government,  liable  for  certain  costs  that  could  have 
been  avoided  by  prudent  contractor  action  (for  example,  the  costs  of 
property  damage  caused  by  contractor  negligence  or  fines  levied  for 
ES&H  violations  caused  by  contractor  negligence).^  The  contractor’s  max¬ 
imum  liability  for  avoidable  costs  will  be  the  fee  earned — both  the  base 
fee  and  the  award  fee — for  the  evaluation  period  during  which  the  con¬ 
tractor  negligence  or  noncompliance  occurred.  The  new  regulations, 
which  will  apply  to  all  new  or  renewed  for-profit  M&o  contracts, 
represent  a  significant  departure  from  doe’s  historical  practice  of  indem¬ 
nifying  its  contractors  against  virtually  all  costs  incurred.  At  the  same 
time,  to  compensate  contractors  for  the  increased  risks  involved,  the 
regulations  substantially  increase  the  total  compensation  for  which  con¬ 
tractors  will  be  eligible.  The  implications  of  the  new  regulations  are  dis¬ 
cussed  further  in  chapter  4. 


^The  final  regulations  adopting  and  modifying  the  interim  regulations  were  promulgated  on  June  19 
1991. 
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Contractor  performance  is  assessed  for  award  purposes  every  6  months 
according  to  a  performance  evaluation  plan.  The  plan  is  to  contain  the 
criteria  to  be  considered  under  each  performance  area  evaluated. 
Various  doe  offices  prepare  and  review  the  evaluation  report  upon 
which  the  award  amount  is  calculated. 


The  performance  evaluation  plans  for  the  Albuquerque  award  fee  con¬ 
tractors  include  performance  objectives  and  evaluation  criteria  for  six 
broad  functional  performance  groups:  (1)  general  management;  (2)  oper¬ 
ations  and  weapons;  (3)  waste  management,  operational  surety,  and 
environmental  restoration;  (4)  environment,  safety,  and  health;  (5)  safe¬ 
guards  and  security;  and  (6)  resources  and  business  management. 
Weights  are  assigned  to  each  group  to  reflect  the  emphasis  that  doe 
wants  to  place  on  that  aspect  of  performance  in  the  award  determina¬ 
tion.  Each  of  these  groups  contains  two  to  eight  functional  performance 
areas.  The  performance  evaluation  plans  are  sent  to  the  contractors 
before  the  start  of  each  evaluation  period. 

Albuquerque  has  area  offices  located  at  each  site  managed  by  award  fee 
contractors.'*  Area  office  staff  gather  and  assess  information  on  con¬ 
tractor  performance  on  a  continuing  basis.  These  offices  prepare  the  ini¬ 
tial  draft  performance  evaluation  plan  with  input  from  the  Albuquerque 
divisions  and  doe  headquarters.  Staff  from  the  Albuquerque  divisions 
have  more  expertise  in  specialized  areas  of  contractor  performance  and 
provide  a  broader  perspective  on  issues  related  to  contractor  perform¬ 
ance.  Over  30  Albuquerque  divisions  are  involved  in  the  award  fee 
administration  process. 

The  plan  is  reviewed  by  the  Albuquerque  Award  Fee  Board,  which  is 
chaired  by  the  Albuquerque  deputy  manager  and  composed  of  senior 
Albuquerque  officials  and  the  area  office  managers.  The  fee  determina¬ 
tion  official — the  Albuquerque  manager — then  concurs  with  the  plan 
and  submits  it  to  doe  headquarters  for  final  approval.  Before  fiscal  year 
1989,  doe  headquarters  did  not  participate  in  the  award  process. 


Award  Determination  6-month  evaluation  period,  Albuquerque  area  offices 

prepare  a  draft  performance  evaluation  report  to  determine  award 

^The  office  located  at  the  WIPP  site  is  called  a  “project  office,”  but  because  it  performs  functions 
similar  to  those  of  an  area  office,  in  this  report  we  refer  to  all  the  on-site  offices  as  “area  offices.” 


Performance  Evaluation 
Plans 


Assessment  of 
Contractor 
Performance  to 
Determine  Awards 
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amounts.  As  with  the  performance  evaluation  plan,  these  offices  con¬ 
sider  input  from  the  Albuquerque  divisions  and  doe  headquarters  in  pre¬ 
paring  the  report,  which  describes  contractor  performance  in  each  of 
the  functional  performance  groups.  Albuquerque  uses  five  categories  in 
describing  contractor  performance.  “Significant  deficiencies”  or  “signifi¬ 
cant  achievements”  are  those  that  are  judged  to  represent  major  events 
or  sustained  levels  of  performance  of  sufficient  importance  to  affect  the 
award  determination.  “Notable  deficiencies”  or  “notable  achievements” 
are  those  that  are  judged  to  be  noteworthy  but  of  lesser  significance.® 
“Observations”  are  positive  or  negative  comments  judged  to  be  worth 
mentioning  but  not  significant  enough  to  affect  the  award 
determination. 

Contractor  performance  in  each  functional  performance  group  is 
assigned  a  numerical  score  and  a  rating  based  on  the  number  and  nature 
of  achievements  and  deficiencies.  The  performance  scale  in  the  new 
award  fee  rule  consists  of  “outstanding”  (numerical  scores  of  96  to  100 
points),  “good”  (86  to  95  points),  “satisfactory”  (76  to  85  points),  “mar¬ 
ginal”  (66  to  75  points)  and  “unsatisfactory”  (65  points  or  less).  “Out¬ 
standing”  performance,  for  example,  substantially  exceeds  expected 
levels,  with  several  significant  or  notable  achievements  and  no  deficien¬ 
cies.  An  overall  rating  is  calculated  by  multiplying  the  numerical  score 
for  each  performance  group  by  the  assigned  weight  for  that  perform¬ 
ance  group  and  totaling  the  results. 

The  performance  evaluation  reports  are  to  address  contractor  perform¬ 
ance  under  each  performance  objective.  If  warranted,  however,  per¬ 
formance  in  areas  other  than  those  covered  by  performance  objectives  is 
also  to  be  addressed.  For  example,  even  though  doe  had  not  established 
a  performance  objective  requiring  strategic  planning,  in  rating  the  wipp 
contractor’s  performance  for  the  first  half  of  fiscal  year  1991,  doe  cited 
the  contractor’s  lack  of  strategic  planning  as  a  deficiency. 

Once  the  reports  are  drafted,  they  are  reviewed  by  the  Albuquerque 
Award  Fee  Board.  The  board  reaches  consensus  on  a  contractor’s  rating 
and  award  amount.  The  Albuquerque  manager  makes  the  final  award 
determination.  Since  fiscal  year  1989,  doe  headquarters  has  also 
reviewed  and  approved  the  award  determination. 


■’’Before  fiscal  year  1991,  Albuquerque  referred  to  performance  in  this  category  simply  as  “achieve¬ 
ments”  and  “deficiencies.” 
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Current  Award  Fee 
Structure 


The  maximum  fee — both  base  amount  and  award  fee — that  a  contractor 
can  earn  is  derived  from  the  estimated  contract  costs  and  the  amount  of 
risk  involved.  Table  1.2  shows  the  contract  costs,  base  fees,  available 


award  fee  pools,  and  actual  awards  paid  to  Albuquerque  contractors  in 
fiscal  year  1990. 

ICIUIC  •  CCAj 

Award  Fee  Pools,  and  Actual  Awards 

Dollars  in  Millions 

Paid  to  Albuquerque  Contractors,  Fiscal 

Awards 

Year  1990 

Facility 

Costs  Base  fee 

Fee  pool 

Paid 

Kansas  City  Plant 

$527 

$2.6 

$12,1 

$8.5 

Pantex  Plant 

182 

1.2 

5.7 

3.1 

Mound  Plant 

195 

1.0 

8.3 

4.4 

Pinellas  Plant 

139 

0.6 

6.3 

3.8 

WIPP 

68 

1.1 

4.5 

1.9 

Objectives,  Scope,  and 
Methodology 


We  recently  implemented  a  special  audit  effort  to  help  ensure  that  areas 
vulnerable  to  fraud,  waste,  abuse,  and  mismanagement  are  identified 
and  that  appropriate  corrective  actions  are  taken.  This  effort  focuses  on 
16  areas,  one  of  which  is  doe’s  contracting  practices.  As  part  of  this 
examination  of  doe  contracting  practices,  the  Chairman,  Senate  Com¬ 
mittee  on  Governmental  Affairs,  requested  that  we  examine  (1)  the 
effectiveness  of  doe’s  use  of  performance  objectives  to  set  expectations 
and  evaluate  contractor  performance,  (2)  the  effectiveness  of  doe’s  use 
of  data  from  on-site  reviews  to  evaluate  contractor  performance  for 
award  fee  purposes,  and  (3)  the  effect  of  doe’s  new  award  fee  regula¬ 
tions  on  the  performance  evaluation  and  award  determination  process. 


As  agreed  with  the  requester,  we  concentrated  our  review  on  the  con¬ 
tractors  operating  under  cost-plus-award-fee  contracts  with  doe’s  Albu¬ 
querque  Field  Office.  We  selected  this  office  because  it  is  responsible  for 
administering  several  large  cost-plus-award-fee  contracts,  including 
those  for  four  defense  materials  production  plants,  as  well  as  the  cost- 
plus-award-fee  contract  for  wipp.  We  have  included  references  to  award 
fee  activities  and  initiatives  at  other  doe  offices  where  appropriate. 


We  obtained  the  views  of  doe  officials  and  analyzed  documents  from  the 
Albuquerque  Field  Office’s  divisions;  the  Amarillo,  Kansas  City,  Dayton, 
and  Pinellas  area  offices;  the  wipp  Project  Office;  and  doe  headquarters. 
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To  determine  the  effectiveness  of  doe’s  use  of  performance  objectives, 
we  reviewed,  analyzed,  and  discussed  with  doe  officials  (1)  the  fiscal 
year  1991  performance  evaluation  plans  for  the  Albuquerque  award  fee 
contractors,  (2)  a  February  1991  report  on  doe  award  fees  submitted  by 
DOE  officials  to  doe  headquarters,  (3)  criteria  related  to  cost-plus-award- 
fee  contracts  in  the  Federal  Acquisition  Regulations,  Department  of 
Energy  Acquisition  Regulations,  and  other  doe  documents,  and  (4)  the 
Albuquerque  award  fee  performance  evaluation  reports  for  fiscal  year 
1990  and  the  first  half  of  fiscal  year  1991. 

To  evaluate  the  effectiveness  of  doe’s  use  of  contractor  performance 
data  from  on-site  reviews,  we  reviewed  a  judgmentally  selected  sample 
of  these  reviews  to  (1)  determine  whether  their  findings  affected  subse¬ 
quent  award  fee  performance  evaluation  reports  and  (2)  identify 
instances  in  which  contractors  had  not  corrected  deficiencies  contained 
in  the  reviews.  We  also  reviewed  contractors’  self-assessments  for  fiscal 
year  1990  to  determine  whether  they  had  addressed  previously  identi¬ 
fied  deficiencies. 

To  determine  the  effect  of  doe’s  new  award  fee  regulations,  we 
examined  the  regulations  (published  as  an  interim  final  rule  on  Feb.  7, 
1991,  and  as  a  final  rule  on  June  19,  1991)  and  the  published  notices 
that  preceded  them.  We  discussed  the  effect  of  the  new  regulations  with 
doe  officials  at  headquarters  and  Albuquerque. 

We  conducted  our  review  between  July  1990  and  July  1991  in  accor¬ 
dance  with  generally  accepted  government  auditing  standards.  We  dis¬ 
cussed  the  contents  of  this  report  with  responsible  agency  officials,  who 
agreed  with  the  factual  information.  We  have  incorporated  their  com¬ 
ments  where  appropriate.  As  requested,  we  did  not  obtain  written 
agency  comments. 
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Many  of  the  objectives  contained  in  Albuquerque’s  performance  evalua¬ 
tion  plans  are  so  broad  and  subjective  that  they  do  not  effectively  com¬ 
municate  what  is  expected  of  the  contractor.  The  performance 
objectives  do  not  specify  measurable  expectations,  standards,  or  mile¬ 
stones  against  which  to  judge  contractor  performance.  As  a  result,  the 
contractor  may  not  have  a  clear  understanding  of  what  level  of  per¬ 
formance  would  be  considered  successful,  and  doe  units  often  disagree 
about  how  specific  performance — either  bad  or  good — should  be 
assessed  for  the  award  determination.  While  doe  appears  to  be  moving 
toward  requiring  more  specific  objectives,  questions  remain  about 
whether  doe’s  actions  will  be  sufficient  to  overcome  its  long-standing 
approach  of  establishing  broad  goals  and  relying  on  its  contractors  to 
plan  and  carry  out  the  specific  programs. 


Many  Performance 
Objectives  Are  Not 
Specific  Enough 


About  half  of  the  Albuquerque  performance  objectives  do  not  tell  the 
contractors  specifically  what  is  expected  of  them  and  what  significance 
will  be  attached  to  specific  accomplishments  or  failures.  Because  the 
objectives  contain  no  criteria,  standards,  or  milestones  against  which  to 
measure  the  contractors’  performance,  the  level  and  significance  of  their 
performance  are  open  to  interpretation. 

In  writing  performance  objectives,  Albuquerque  often  uses  broad  terms 
such  as  “improving  programs’’  or  “fulfilling  conceptual  goals”  but  does 
not  specify  what  is  to  be  accomplished  or  how  it  will  be  measured.  For 
example,  in  the  first  half  of  fiscal  year  1991,  three  of  the  six  perform¬ 
ance  objectives  established  for  the  Mound  Plant  contractor  in  the  critical 
ES&H  area  were:  (1)  “enhance  the  Industrial  Hygiene  Program,”  (2) 
“enhance  the  Health  Physics  Program,”  and  (3)  “enhance  and  accelerate 
the  Safety  Analysis  Program.” 

Other  performance  objectives  represent  broad  conceptual  goals.  For 
example,  for  the  first  half  of  fiscal  year  1991,  an  objective  established 
for  the  Kansas  City  Plant  contractor  was  to  “ensure  that  sound  business 
management  and  resources  are  being  directed  and  utilized  on  those 
projects  and  activities  most  critical  to  doe’s  mission.”  Such  an  objective 
conveys  no  criteria  or  standards  against  which  performance  can  be 
measured. 
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Lack  of  Specificity 
Results  in  Confusion 
in  Judging  Contractor 
Performance 


Some  DOE  officials  have  recognized  the  problems  caused  by  the  lack  of 
specificity  in  performance  expectations.  In  a  February  1991  report  to 
the  Assistant  Secretary  for  Defense  Programs,  a  doe  task  force  recom¬ 
mended  improvements  to  the  award  determination  process  largely 
because  of  the  misunderstandings  caused  by  the  subjectivity  of  the 
process. 


The  February  report  resulted  from  a  study  of  the  award  fee  process  by  a 
DOE  group  composed  of  representatives  from  contractors,  doe  headquar¬ 
ters,  and  DOE  field  offices.  The  report  noted  that  because  doe’s  perform¬ 
ance  evaluation  plans  are  essentially  subjective,  there  is  a  low  level  of 
understanding  between  doe  field  offices  and  the  contractors  and 
between  field  offices  and  headquarters  on  the  meaning  and  intent  of 
performance  objectives.  The  lack  of  specificity  frequently  leads  to  dis¬ 
agreements  both  on  what  constitutes  an  accomplishment  and  on  what 
significance  the  accomplishment  has.  The  report  also  noted  that  contrac¬ 
tors  see  the  expectation  setting  and  evaluation  processes  as  arbitrary 
and  ambiguous  and  often  complain  that  they  do  not  know  what  doe  per¬ 
ceives  as  successful  performance.  As  a  result,  the  process  often  does  not 
effectively  provide  doe’s  contractors  with  an  incentive  to  excel. 

The  report  noted  that  doe  personnel  who  determine  the  contractor’s 
rating  and  award  at  the  end  of  the  evaluation  period  often  disagree  on 
how  the  contractor’s  performance  should  be  interpreted  and  reflected  in 
the  rating.  For  example,  some  headquarters  reviewers  put  different 
emphases  on  certain  events  that  occurred  during  the  rating  period  than 
field  reviewers  did.  The  report  stated  that  contractors  would  have  a 
greater  opportunity  to  be  responsive  if  doe  directions  were  clear  and 
consistent. 


Our  analysis  also  indicated  frequent  disagreement  among  doe  officials 
about  what  constitutes  a  contractor  deficiency  for  award  determination 
purposes.  Performance  judgments  that  the  Albuquerque  division  offi¬ 
cials,  who  have  technical  expertise  in  specific  areas  of  contractor  per¬ 
formance,  provided  as  input  to  the  contractors’  evaluation  reports  were 
often  not  upheld  by  area  office  or  Award  Fee  Board  officials.  For 
example,  in  fiscal  year  1990,  fewer  than  half  of  the  contractors’  actions 
described  by  Albuquerque  division  officials  as  “significant  deficiencies” 
were  reflected  as  such  in  the  final  performance  evaluation  reports.  Most 
of  the  remainder  were  downgraded  to  “deficiencies”;  a  few  were  omitted 
altogether.  Likewise,  fewer  than  half  of  the  contractors’  actions 
described  by  Albuquerque  division  officials  as  “deficiencies”  appeared 
as  such  in  the  performance  evaluation  reports.  A  small  percent  of  the 


Page  16 


GAO/RCED-92-9  DOE’s  Contractor  Appraisal  Process 


Chapter  2 

DOE  Performance  Evaluation  Plans  Do  Not 
Adequately  Convey  Expectations 


remainder  were  downgraded  to  “observations”;  the  rest  were  omitted. 
The  area  offices  made  most  of  the  decisions  to  downgrade  or  omit  these 
deficiencies. 

Furthermore,  views  differed  between  doe  headquarters  and  Albu¬ 
querque  officials.  For  example,  Albuquerque  officials  considered  the 
wipp  contractor’s  progress  on  the  Final  Safety  Analysis  Report  during 
the  first  half  of  fiscal  year  1990  to  be  an  “achievement”;  doe  headquar¬ 
ters  officials  judged  the  contractor’s  performance  on  the  report  to  be  a 
“deficiency.” 

To  resolve  such  differences,  the  February  1991  task  force  report  identi¬ 
fied  the  need  for:  (1)  more  specificity  in  performance  evaluation  plans 
and  (2)  advance  agreement  between  doe  organizations  and  contractors 
on  how  the  achievement  of  objectives  will  affect  performance  ratings. 
The  report  also  recommended  building  more  objectivity  into  the  per¬ 
formance  evaluation  plans  and  including  performance  indicators  that 
are  as  specific  as  possible. 

Albuquerque  is  also  taking  actions  to  improve  performance  objectives. 

In  a  June  1991  memorandum  on  the  development  of  performance  objec¬ 
tives  for  the  first  half  of  fiscal  year  1992,  Albuquerque  officials  called 
for  objective  performance  requirements  to  the  extent  possible.  As  an 
example,  they  said  that  “enhance  safety  program”  would  be  an  unac¬ 
ceptable  objective,  suggesting  instead  an  objective  such  as  “demonstrate 
material  improvement  in  the  safety  program  by  providing  two  weeks’ 
training  in  fire  protection  per  person.  Certificates  of  training  completion 
will  evidence  accomplishment  of  this  [objective].” 


Improved 
Performance 
Objectives  Will  Be 
Difficult  to  Achieve 


While  doe  is  endorsing  the  use  of  more  specific  performance  objectives, 
this  improvement  will  be  difficult  to  achieve  without  a  change  in  doe’s 
historical  relationship  with  its  contractors — a  change  in  doe’s  basic  cul¬ 
ture.  As  noted,  top  doe  officials  agree  that  more  specific  performance 
objectives  are  needed,  but  some  doe  officials  continue  to  believe  that 
performance  objectives  should  be  subjective.  This  view  reflects  doe’s 
traditional  philosophy  with  regard  to  contractor  activities — that  is,  doe 
establishes  broad  objectives  and  relies  on  the  contractors  to  plan  and 
carry  out  the  programs  with  little  doe  oversight.  We  have  characterized 
this  philosophy  as  one  of  “least  interference”  in  contractor  operations. 

According  to  the  February  1991  task  force  report,  some  doe  officials 
believe  that  doe  should  not  rely  too  heavily  on  specific  performance 
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objectives  because  doing  so  would  reduce  the  flexibility  the  contractors 
need  to  effectively  manage  doe  facilities.  If  doe  were  to  establish  spe¬ 
cific,  measurable  performance  requirements,  the  officials  contend,  the 
contractors  might  focus  most  of  their  efforts  on  these  requirements,  at 
the  expense  of  broader  goals  such  as  maintaining  basic  systems  and 
processes  and  responding  to  unanticipated  events.  This  view  was  repre¬ 
sented  in  a  December  1990  memorandum  from  a  Deputy  Assistant  Sec¬ 
retary  for  Defense  Programs,  who  concluded  that  the  use  of  broader 
goals  and  standards  had  resulted  in  the  contractors’  initiating  and  imple¬ 
menting  several  valuable  programs  at  the  sites. 

The  task  force  report  noted  that  a  second  argument  for  broader  goals  is 
that  DOE  hires  the  contractors  in  part  for  their  management  expertise. 
Accordingly,  part  of  the  evaluation  must  address  how  well  the  con¬ 
tractor  establishes  priorities  to  best  use  the  resources  available  to 
accomplish  the  range  of  requirements  under  the  contract,  doe  must  sub¬ 
jectively  assess  the  contractor’s  performance  in  accomplishing  this.  One 
Albuquerque  division  director,  for  example,  commented  that  broad 
objectives  are  appropriate.  If  doe  had  to  develop  more  specific  objec¬ 
tives  for  the  program,  he  said,  doe  would  not  need  the  contractor — it 
could  run  the  program  itself.  An  area  office  branch  chief  made  similar 
comments. 


Conclusions  broad  performance  objectives  doe  currently  uses  to  assess  the  per¬ 

formance  of  its  award  fee  contractors  lead  to  disagreements  among  doe 
units  on  how  contractor  performance  should  be  rated.  Contractors  are 
also  dissatisfied  because  they  view  the  process  as  arbitrary  and 
ambiguous. 

While  DOE  has  recently  endorsed  the  need  for  more  specific  and  measur¬ 
able  objectives,  implementing  this  change  will  be  difficult  because  of 
philosophical  differences  within  doe  about  the  appropriate  relationship 
with  the  contractors.  Proactive  and  strong  leadership  will  be  needed  to 
overcome  institutional  resistance  to  changing  the  historical  relationship 
between  doe  and  its  contractors. 


RcCOinrQGridutionS  recommend  that  the  Secretary  of  Energy  ensure  continued  improve¬ 

ment  in  the  performance  evaluation  and  award  determination  processes 
by  (1)  requiring  that  specific,  measurable  performance  objectives  be 
developed,  (2)  developing  procedures  to  communicate  to  the  contractor 
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what  is  expected  and  how  it  will  be  measured,  and  (3)  establishing  pro¬ 
cedures  to  ensure  that  these  requirements  are  implemented  consistently 
by  all  DOE  organizations  that  take  part  in  the  award  fee  process. 
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Although  Albuquerque  considers  a  voluminous  amount  of  information 
from  every  office  involved,  questions  remain  about  whether  all  of  the 
serious  performance  deficiencies  identified  during  routine  oversight  are 
appropriately  evaluated  as  part  of  the  award  fee  process.  In  addition, 
doe’s  performance  evaluations  do  not  routinely  consider  the  timeliness 
and  thoroughness  of  a  contractor’s  actions  to  address  review  findings  or 
deficiencies  identified  in  previous  performance  evaluations.  Further,  doe 
has  not  effectively  used  a  new  award  fee  requirement — the  contractor’s 
self-assessment — to  evaluate  the  contractor’s  response  to  deficiencies 
identified  in  previous  performance  evaluations. 


Some  Performance 
Information  Not 
Appropriately 
Reflected  in 
Performance 
Evaluation  Reports 


In  compiling  award  fee  performance  evaluation  reports,  doe  uses  con¬ 
tractor  reports,  performance  indicators  for  various  programs,  evalua¬ 
tions  of  contractor  products,  and  on-site  reviews  of  specific  contractor 
functions.  According  to  doe  officials,  management  must  evaluate  the  sig¬ 
nificance  of  such  information  in  deciding  whether  the  performance 
information  warrants  inclusion  in  the  performance  evaluation.  The  peri¬ 
odic  on-site  reviews — such  as  the  reviews  of  the  contractors’  procure¬ 
ment  systems,  property  management  systems,  and  financial  systems — 
are  key  because  they  are  based  on  direct  observation  and  evaluation  by 
functional  experts. 

In  determining  the  award  amount,  Albuquerque  managers  consider  how 
many  achievements  and  deficiencies  the  contractor  has  received.  Our 
analysis  showed  that  some  on-site  reviews  disclosed  deficiencies  that 
did  not  appear  to  be  treated  appropriately  in  the  contractor’s  perform¬ 
ance  evaluation  report.  One  example  comes  from  a  Contractor 
Purchasing  System  Review  (cpsr)  at  wipp.  cpsrs  are  on-site  reviews  con¬ 
ducted  periodically  by  Albuquerque  personnel  to  ensure  that  contrac¬ 
tors’  purchasing  systems  comply  with  doe  and  federal  acquisition 
regulations.  These  purchasing  systems  and  the  periodic  reviews  are 
important  because,  in  total,  the  five  Albuquerque  award  fee  contractors 
annually  award  subcontracts  totaling  nearly  $400  million. 

A  1989  cpsr  found  deficiencies  in  wipp’s  purchasing  system.  Because  of 
the  seriousness  of  these  deficiencies,  doe  reduced  the  contractor’s 
threshold  for  procurement  actions  needing  prior  doe  approval  from  $25 
million  (the  threshold  given  to  most  Albuquerque  contractors)  to 
$150,000.  In  the  contractor’s  performance  evaluation  for  that  period, 
however,  the  only  mention  of  the  cpsr  was  an  observation  that  “while 
the  procurement  function  has  supported  the  program,  preliminary  indi¬ 
cations  show  that  there  are  a  number  of  areas  needing  improvement. 
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such  as:  lack  of  advance  procurement  planning;  deficiencies  in  updating 
and  implementing  procedures;  and  small  business  set-aside  reporting 
practices.”  Because  observations  in  the  award  fee  report  do  not  affect 
the  overall  rating  or  the  award  determination,  the  contractor  was  not 
penalized  for  these  serious  procurement  management  problems. 

Not  only  was  the  wipp  contractor  not  penalized,  it  received  an  “achieve¬ 
ment”  in  its  performance  evaluation  the  following  year  after  a  follow-up 
CPSR  showed  significant  improvements.  According  to  the  evaluation 
report,  “considerable  effort  had  been  applied  to  respond  to  the  previous 
34  recommendations  and  to  increasing  the  overall  effectiveness  of  the 
procurement  systems.”  In  effect,  because  “achievements”  increase  the 
award  amount,  the  wipp  contractor  actually  benefited  from  having  a 
deficient  procurement  system. 

Albuquerque  officials,  however,  assert  that  they  have  to  view  the  con¬ 
tractor’s  performance  in  total.  According  to  an  Albuquerque  official, 
individual  deficiencies  such  as  the  problems  with  the  wipp  contractor’s 
procurement  system  did  not  merit  greater  significance  when  compared 
with  the  contractor’s  overall  performance  in  other  areas  of  procurement 
management  such  as  progress  in  socioeconomic  goals. 


Contractors’  Actions 
to  Address 
Deficiencies  Not 
Always  Considered  in 
Award  Determinations 


Although  DOE  procedures  require  that  performance  evaluations  consider 
a  contractor’s  responsiveness  to  on-site  reviews  and  performance  evalu¬ 
ations,  Albuquerque’s  evaluations  have  not  done  so  consistently.  First, 
because  not  all  divisions  have  developed  procedures  and  systems  for 
tracking  contractors’  responses  to  review  findings,  a  contractor’s  failure 
to  respond  might  not  be  reflected  in  the  award  determination.  Second, 
Albuquerque  hcis  not  ensured  that  contractors’  self-assessments  system¬ 
atically  report  actions  taken  on  deficiencies  cited  in  previous  perform¬ 
ance  evaluations. 


Contractors’  Lack  of 
Action  on  Review  Findings 
Not  Reflected  in 
Performance  Evaluations 


Six  of  the  10  Albuquerque  divisions  whose  review  procedures  we 
examined  have  not  established  central  systems  to  track  contractors’ 
responses  to  review  findings.  Three  of  the  divisions  have  not  established 
written  procedures  specifying  when  and  how  contractors  should 
address  recommendations  from  the  on-site  reviews.  Consequently, 
because  a  contractor’s  failure  to  respond  to  review  findings  may  not  be 
noted  in  a  timely  manner,  the  contractor’s  failure  to  address  previous 
recommendations  might  not  be  considered  in  the  award  determination. 
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For  example,  on  the  basis  of  a  March  1990  personal  property  review,  the 
Pinellas  contractor  received  a  “significant  achievement”  in  its  perform¬ 
ance  evaluation  for  the  second  half  of  fiscal  year  1990  to  recognize  its 
development  of  an  effective,  automated  equipment  tracking  system.  But 
neither  the  property  review  nor  the  performance  evaluation  report 
revealed  that  Albuquerque  had  not  ensured  that  corrective  action  had 
been  taken  on  14  of  15  recommendations  it  made  during  the  contractor’s 
1985  personal  property  review.  At  that  time,  the  Albuquerque  property 
management  program  did  not  have  a  central  tracking  system  to  monitor 
actions  on  outstanding  recommendations.  Such  a  tracking  system  has 
since  been  developed. 

Similarly,  the  Kansas  City  contractor’s  award  determination  for  per¬ 
formance  during  the  first  half  of  1990  was  not  affected  by  the  con¬ 
tractor’s  failure  to  implement  recommendations  made  over  3  years 
earlier.  According  to  a  1990  follow-up  financial  systems  review,  the  con¬ 
tractor  had  not  yet  taken  action  to  correct  a  problem  identified  in  a 
review  conducted  in  1987.  In  response  to  the  earlier  review,  the  con¬ 
tractor  had  agreed  to  develop  written  procedures  to  ensure  that  costs 
for  projects  being  done  for  other  governmental  agencies  were  not 
incurred  before  the  funds  were  received.  Albuquerque’s  financial  man¬ 
agement  program  does  not  have  a  central  tracking  system  to  monitor 
action  on  outstanding  recommendations. 


Albuquerque  Has  Not 
Ensured  That  Contractors 
Systematically  Report  the 
Status  of  Previous 
Deficiencies 


Albuquerque  has  not  ensured  that  contractors  systematically  report 
actions  taken  to  correct  each  “significant  deficiency”  and  “deficiency” 
cited  in  previous  performance  evaluations.  Although  doe  instituted,  in 
fiscal  year  1990,  a  new  self-assessment  procedure  in  which  contractors 
are  to  report  the  status  of  previously  identified  deficiencies,  contractor 
self-assessments  have  generally  not  included  this  information. 


Albuquerque  officials  can  raise  or  lower  the  contractor’s  overall  rating 
as  a  result  of  their  evaluation  of  the  adequacy  of  the  contractor’s  self- 
assessment.  Although  the  Albuquerque  contractors’  self  assessments 
have  generally  not  reported  the  status  of  previous  deficiencies,  Albu¬ 
querque  has  not  penalized  the  contractors.  According  to  an  Albuquerque 
official,  they  wanted  to  give  contractors  time  to  understand  the  new 
self-assessment  requirement.  In  contrast,  Albuquerque  did  reward  the 
Pantex  Plant  contractor  for  its  self-assessment  for  the  first  half  of  fiscal 
year  1990.  Albuquerque  considered  the  self-assessment  to  be  especially 
well  structured,  realistic,  and  candid.  As  a  result,  Albuquerque 
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increased  the  contractor’s  overall  rating  by  one  point,  which  increased 
the  contractor’s  award  by  about  $125,000. 


Conclusions 


Recommendations 


Because  on-site  reviews  are  based  on  direct  observation  and  evaluation 
by  functional  experts,  such  reviews  are  a  key  tool  for  evaluating  con¬ 
tractor  performance.  Albuquerque  discounted  serious  deficiencies 
revealed  by  some  reviews,  however,  in  determining  the  contractor’s 
award  fee.  Because  of  the  value  of  the  information  provided  by  these 
on-site  reviews,  appropriate  consideration  should  be  given  to  review 
findings  in  making  award  determinations. 

In  addition,  Albuquerque  does  not  always  consider  a  contractor’s 
response  to  review  findings  in  performance  evaluations  and  award 
determinations.  Procedures  specifying  when  and  how  the  contractors 
should  address  review  recommendations  and  systems  to  track  contrac¬ 
tors’  corrective  actions  are  needed  to  ensure  that  the  contractor’s 
responsiveness  is  monitored  and  evaluated  as  part  of  the  performance 
evaluation  for  determining  the  award  fee. 

Similarly,  Albuquerque  lacks  a  system  to  monitor  contractors’  corrective 
actions  for  “deficiencies”  identified  in  previous  performance  evalua¬ 
tions.  The  contractor’s  self-assessments  cannot  be  relied  on  for  this  pur¬ 
pose  because  Albuquerque  has  not  fully  enforced  the  requirement  to 
ensure  that  all  deficiencies  identified  in  performance  evaluations  are 
appropriately  addressed. 


We  recommend  that  the  Secretary  of  Energy  (1)  ensure  that  the  results 
of  on-site  reviews  are  given  appropriate  consideration  in  the  award 
determination  process,  (2)  establish  procedures  for  requiring  and 
tracking  contractor  responses  to  all  review  findings,  (3)  ensure  that  con¬ 
tractor  responses  to  previous  review  findings  are  considered  in  all  on¬ 
site  reviews,  and  (4)  establish  procedures  for  ensuring  that  all  deficien¬ 
cies  identified  in  performance  evaluations  are  addressed  in  contractor 
self-assessments. 
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Because  of  recognized  deficiencies  in  its  contractor  management  proce¬ 
dures,  DOE  has  issued  a  new  rule  to  make  the  profit-making  contractors 
accountable  for  certain  costs  previously  borne  by  doe  and,  at  the  same 
time,  to  significantly  increase  the  potential  award  amounts  that  contrac¬ 
tors  can  earn.  All  new  or  existing  contracts  that  are  extended  after 
March  11, 1991,  are  subject  to  the  new  rule. 

Implementing  the  rule,  however,  will  be  a  formidable  task,  as  illustrated 
by  the  large  number  and  scope  of  the  issues  doe  itself  has  said  must  be 
addressed  to  effectively  implement  the  rule.  Questions  remain  about 
whether  doe  will  be  in  a  position  to  hold  the  contractors  accountable  for 
avoidable  costs.  The  potential  exists,  therefore,  for  contractors  to 
receive  considerable  increases  in  fees  without  actually  being  subject  to 
the  increased  accountability  requirements  of  the  new  rule. 


New  Rule  Issued  to 
Increase  Contractor 
Accountability 


Historically,  doe  has  indemnified  its  profit-making  management  and 
operating  (m&o)  contractors  against  nearly  all  risks  and  costs;  doe  has 
rarely  disallowed  any  costs  incurred  by  the  contractors.  For  example, 
doe  reimbursed  its  contractors  for  fines  and  penalties  imposed  for  viola¬ 
tions  of  federal  and  state  environmental  regulations  unless  the  fines  or 
penalties  were  the  result  of  willful  misconduct  or  lack  of  good  faith  on 
the  part  of  a  few  key  contractor  personnel.* 

To  improve  its  fiscal  controls  over  its  contractors,  doe  developed  a  new 
accountability  rule.  Under  this  rule,  doe  will  (1)  hold  the  contractors, 
rather  than  the  government,  responsible  for  costs  that  could  have  been 
avoided  by  proper  contract  performance  and  (2)  increase  contractors’ 
potential  fees  to  offset  the  increase  in  their  financial  risk. 

The  rule  will  apply  to  all  new  profit-making  M&o  contracts.  Under  the 
rule,  the  contractor’s  liability  is  no  longer  limited  to  the  willful  actions 
of  a  few  key  employees,  but  covers  the  negligence  and  misconduct  of  all 
employees.  Contractors  will  be  liable  for  all  negligence  and  misconduct 
that  result  in  civil  fines  and  penalties,  destruction  or  loss  of  government 
property,  and  other  avoidable  costs.  The  rule  allows  the  doe  contracting 
officer  to  make  the  final  decision  on  whether  or  not  to  reimburse  the 


’Under  DOE  regulations,  costs  incurred  in  defense  of  any  civil  or  criminal  fraud  proceeding  brought 
by  the  government  against  a  contractor  are  unallowable  if  the  proceeding  results  in  a  conviction. 
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contractor  for  such  costs.  The  contractor’s  total  financial  liability,  how¬ 
ever,  is  limited  to  the  amount  of  the  fee  actually  earned  during  the  eval¬ 
uation  period  in  which  the  contractor’s  negligence  or  misconduct 
occurred. 

To  compensate  the  contractors  for  their  potential  increased  liability,  the 
new  rule  substantially  increases  the  total  fee  available  to  the  contrac¬ 
tors.  Table  4.1  compares  the  Albuquerque  contractors’  fiscal  year  1991 
base  fees  and  available  award  fee  pools  with  the  amounts  that  would 
have  been  available  under  the  new  award  fee  provisions. ^  As  shown,  the 
total  fees  available  to  the  contractors  would  have  increased  substan¬ 
tially,  particularly  for  the  base  fee  portion.  The  increases  in  the  base  fee 
are  intended  not  only  to  adjust  for  inflation  since  the  last  revision  in 
1983,  but  also  to  reflect  the  increased  accountability  and  risk  related  to 
operating  a  facility. 


ICIUI9  I*  ovi  1 1^01  loul  1  ui  r  ia\.rCii  iccii 

1991  Fees  Available  to  Albuquerque 

Dollars  in  Millions 

Contractors  With  Potential  Fees 

Current  fee  New  fee 

Percent 

Available  Under  New  Rule 

Facility 

Base  Award  Total  Base  Award 

Total  increase 

Kansas  City  Plant 

$1.26 

$14.17 

$15.43 

$8.40 

$12.60 

$21.00 

36 

Pantex  Plant 

1.41 

6.57 

7.98 

4.70 

9.40 

14.10 

77 

Mound  Plant 

.58 

10.47 

11.05 

5.82 

8.73 

14.55 

32 

Pinellas  Plant 

.61 

6.89 

7.50 

4.05 

6.08 

10.13 

35 

WIPP 

1.20 

3.60 

4.80 

3.00 

4.50 

7.50 

56 

Although  the  base  fee  has  historically  been  considered  the  contractors’ 
primary  compensation  for  satisfactory  performance,  receipt  of  the 
entire  base  fee  is  no  longer  guaranteed.  Under  the  new  rule  the  contrac¬ 
tors  will  have  to  refund  up  to  50  percent  of  the  base  fee  if  their  overall 
performance  is  rated  “marginal”  or  “unsatisfactory.” 


^To  avoid  confusion,  we  use  the  term  “base  fee”  to  refer  to  the  fixed  portion  of  the  fee.  Under  the 
new  rule,  however,  this  portion  of  the  fee  is  called  the  “basic”  fee. 
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DOE  Faces  a  Major 
Challenge  in 
Effectively 

Implementing  the  New 
Accountability  Rule 


To  achieve  the  intended  increase  in  contractor  accountability,  doe  must 
(1)  incorporate  the  new  requirements  into  doe’s  29  cost-plus-award-fee 
contracts,  (2)  develop  day-to-day  operational  procedures  by  which  to 
identify  all  avoidable  costs  and  hold  the  contractors  accountable  for 
them,  and  (3)  train  the  staff  to  implement  the  accountability  procedures 
in  a  timely  manner.  If  doe’s  efforts  in  these  areas  are  not  effective,  the 
contractors  may  receive  the  larger  award  fees  without  actually  incur¬ 
ring  any  additional  liability. 

Two  areas  pose  particular  challenges  to  doe:  identifying  contractor 
accountability  for  loss  or  damage  to  government  property  and  identi¬ 
fying  avoidable  costs.  Accountability  for  lost  or  damaged  property  will 
be  difficult  and  time-consuming  to  determine  because  procedures  will 
have  to  be  developed  to  (1)  determine  the  condition  of  existing  equip¬ 
ment,  (2)  identify  whether  and  how  much  property  was  lost  or  dam¬ 
aged,  (3)  determine  that  damage  was  caused  by  negligence  or 
misconduct  rather  than  routine  wear  and  tear,  and  (4)  identify  the  value 
of  the  property  in  question,  doe  does  not  currently  perform  any  of  these 
functions  as  part  of  its  routine  operations.  For  example,  property  loss  or 
damage  would  normally  be  identified  by  comparing  physical  inventories 
with  property  records.  To  implement  the  accountability  rule,  doe  would 
have  to  periodically  verify  the  accuracy  of  the  physical  inventories  con¬ 
ducted  by  the  contractors.  Without  such  verification,  doe  would  rely 
totally  on  information  provided  by  a  contractor  to  determine  the  con¬ 
tractor’s  liability  for  lost  or  damaged  property.  Albuquerque’s  current 
reviews  of  contractors’  property  management  systems  (which  at  best 
are  scheduled  every  2  years),  however,  focus  on  the  contractors’  policies 
and  procedures;  the  reviews  do  not  include  verification  of  physical 
inventories.  Similarly,  Albuquerque  currently  has  no  process  for  deter¬ 
mining  what  damage  results  from  routine  wear  and  tear  or  for  identi¬ 
fying  the  current  value  of  the  contractor  property. 

The  issue  of  avoidable  costs — costs  incurred  as  the  result  of  negligence 
or  willful  misconduct  by  a  contractor  or  its  subcontractors — is  even  less 
well  defined.  In  addition  to  the  difficulty  of  identifying  instances  in 
which  funds  were  not  used  effectively,  doe  would  also  have  to  decide 
how  much  of  the  costs  incurred  were  actually  avoidable. 
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DOE  Efforts  to 
Implement 
Accountability 
Requirements 


DOE  has  initiated  efforts  to  implement  the  new  rule,  doe’s  March  1991 
contract  for  the  operation  of  the  Oak  Ridge  plants  and  facilities  incorpo¬ 
rates  the  provisions  of  the  new  rule  but  specifies  that  the  new  accounta¬ 
bility  and  increased  compensation  provisions  would  not  go  into  effect 
until  October  1, 1991.  doe  had  much  to  accomplish  by  that  date,  doe  is 
also  negotiating  to  incorporate  provisions  of  the  new  rule  in  several  con¬ 
tracts  that  were  extended  while  the  rule  was  being  finalized,  doe  will 
incorporate  the  provisions  in  the  contracts  with  the  other  profit-making 
contractors  when  their  current  contracts  expire. 

DOE  has  identified  the  following  three  major  issues  that  will  have  to  be 
addressed  in  implementing  the  accountability  requirements: 

The  way  doe’s  instructions  are  communicated  to  the  contractors. 
Because  contractors  would  be  able  to  deny  accountability  for  actions 
undertaken  at  doe’s  direction,  doe  officials  have  recognized  that  they 
must  clarify  who  has  the  authority  to  give  instructions  and  how  these 
instructions  are  to  be  communicated  to  the  contractor. 

Responsibility  for  identifying  avoidable  costs.  According  to  doe  officials, 
this  identification  must  become  a  key  responsibility  of  the  doe  program 
staff,  rather  than  solely  an  administrative  function,  because  only  the 
program  staff  will  have  the  knowledge  to  identify  instances  in  which 
costs  could  have  been  avoided. 

The  need  for  extensive  training  of  doe  staff.  For  several  years  doe  has 
recognized  the  need  to  improve  the  technical  capabilities  of  its  staff,  doe 
believes  training  is  even  more  important  under  the  new  accountability 
rule,  because  implementing  the  rule  effectively  will  require  more  precise 
instructions  to  the  contractors  and  the  technical  capability  to  identify 
avoidable  costs. 

Although  these  issues  are  all  critical  to  the  effectiveness  of  the  new 
accountability  rule,  doe  has  a  limited  number  of  staff  to  monitor  and 
verify  contractor  actions.  According  to  Albuquerque  officials,  their  cur¬ 
rent  level  of  staffing  will  make  it  very  difficult  to  properly  perform 
duties  related  to  implementing  the  accountability  rule  without  a  severe 
impact  on  other  work  load  requirements.  Although  supplemented  by 
some  area  office  staff  members,  Albuquerque  divisions  have  limited 
staff  to  enforce  the  accountability  requirements.  For  example,  Albu¬ 
querque  has  only  four  property  management  positions  dedicated  to 
overseeing  an  estimated  $6  billion  worth  of  government-owned  personal 
property  being  held  by  nine  different  M&o  contractors.  Additionally,  to 
implement  the  accountability  rule,  doe  would  have  to  ensure  that  the 
contractors  perform  appropriate  maintenance  on  government-owned 
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equipment  to  prevent  unnecessary  damage.  But  Albuquerque  has  only 
three  positions  dedicated  to  overseeing  contractor  compliance  with  doe 
maintenance  requirements.  Albuquerque  does  not  yet  know  what  equip¬ 
ment  needs  to  be  maintained  or  what  maintenance  schedules  are  appro¬ 
priate  because  contractors  were  only  recently  required  to  document 
what  equipment  they  have  and  to  develop  maintenance  schedules.  Fur¬ 
ther,  Albuquerque  has  only  five  positions  dedicated  to  overseeing  nearly 
300,000  procurement  actions,  representing  nearly  $2  billion,  performed 
by  its  nine  M&o  contractors  annually.  With  such  a  small  staff  it  will  be 
very  difficult  for  Albuquerque  to  determine  whether  contractors’  pro¬ 
curement  actions  incurred  avoidable  costs. 

The  changes  required  for  doe  to  identify  and  communicate  its  require¬ 
ments  and  hold  the  contractors  accountable  cannot  be  accomplished 
quickly.  For  example,  a  current  doe  effort  to  use  task  orders  to  identify 
specific  tasks  and  milestones  to  be  accomplished  with  authorized  funds 
is  one  way  to  better  communicate  doe’s  instructions  to  the  contractors 
and  hold  them  accountable  for  specific  results.  Officials  responsible  for 
developing  this  approach,  however,  say  that  full  implementation  at  all 
DOE  contractors  should  take  from  3  to  5  years.  This  effort  will  also 
require  extensive  training  for  the  doe  staff  involved. 

In  response  to  the  accountability  rule,  task  order  administration,  and 
other  DOE  business  management  initiatives,  Albuquerque  has  requested 
a  total  of  92  additional  staff  for  fiscal  years  1992  and  1993. 


Conclusions 


doe’s  efforts,  including  the  issuance  of  the  new  rule  to  hold  contractors 
accountable  for  actions  within  their  control,  are  commendable  but  sub¬ 
stantial  additional  effort  will  be  required  to  achieve  doe’s  goals,  doe  has 
coupled  the  increased  accountability  with  increased  award  fees  and 
capped  the  potential  financial  accountability  of  the  contractors  to  make 
the  new  requirements  more  acceptable  to  the  contractors. 

To  actually  achieve  this  increased  contractor  accountability,  however, 
doe  must  develop  day-to-day  operational  procedures  that  will  identify 
avoidable  costs  and  hold  the  contractors  accountable  for  them.  Two 
areas  in  particular  will  be  difficult  and  time-consuming  to  implement: 
accountability  for  loss  or  damage  to  government  property  and  avoidable 
costs.  These  will  require  the  development  of  new  procedures,  additional 
training  for  doe  staff,  and  possibly  the  addition  of  more  staff. 
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The  scope  of  the  changes  needed  in  doe  procedures  and  operations  to 
implement  the  accountability  requirements  raises  the  possibility  that 
contractors  will  be  eligible  for  the  increased  compensation  before  doe  is 
in  position  to  impose  the  accountability  requirements. 


We  recommend  that  the  Secretary  of  Energy  ensure  that  procedures  for 
determining  contractor  liability  for  lost  or  damaged  property  and  avoid¬ 
able  costs  are  in  place  and  that  sufficiently  trained  staff  are  available  to 
implement  the  new  accountability  requirements,  or  delay  the  planned 
increase  in  award  fees  until  doe  is  ready  to  hold  the  contractors 
accountable. 
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